Image adjustment apparatus and image sensor for synchronous image and asynchronous image

ABSTRACT

An image adjustment apparatus includes a receiver which is configured to receive a first input image of an object which is time-synchronously captured and a second input image in which a motion event of the object is sensed time-asynchronously, and an adjuster which is configured to adjust the first input image and the second input image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of U.S. application Ser.No. 17/554,826, filed Dec. 17, 2021, which is a Continuation applicationof U.S. application Ser. No. 16/859,598, filed Apr. 27, 2020, which is aContinuation application of U.S. application Ser. No. 16/279,565 filedon Feb. 19, 2019, in the U.S. Patent and Trademark Office, which is aContinuation application of U.S. application Ser. No. 14/299,599 filedon Jun. 9, 2014, in the U.S. Patent and Trademark Office, now U.S. Pat.No. 10,237,506 issued on Mar. 19, 2019, which claims priority fromKorean Patent Application No. 10-2013-0068855, filed on Jun. 17, 2013,in the Korean Intellectual Property Office, the disclosures of which areincorporated by reference herein in their entireties.

BACKGROUND 1. Field

Exemplary embodiments relate to an image adjustment apparatus and animage sensor. In particular, exemplary embodiments relate to atechnology for adjusting an image and a technology for sensing an image.

2. Description of Related Art

A related art complementary metal-oxide semiconductor (CMOS) imagesensor has been widely used in various devices, for example, smartphonesand camcorders. A related art image sensor may be used for capturing animage or a video, and for providing a user interface for an input objectrecognition and providing an application service such as a video call.

The related art CMOS image sensor adopts a frame-based synchronoussensing structure. For example, the related art CMOS image sensor mayprovide a signal having a predetermined period to a horizontal scanneror a vertical scanner so that image information of each pixel may bereadout and reset. The predetermined period may be referred to as aframe.

SUMMARY

According to an aspect of an exemplary embodiment, there is provided animage adjustment apparatus including a receiver which is configured toreceive a first input image of an object which is time-synchronouslycaptured and a second input image in which a motion event of the objectis sensed time-asynchronously and an adjuster which is configured toadjust the first input image and the second input image.

The adjuster may include a time adjuster which is configured to match anevent timing in the second input image with one of a plurality of frameperiods of the first input image such that time information of the firstinput image and time information of the second input image is adjusted.

The adjuster may also include a space adjuster which is configured toadjust space information of the first input image and space informationof the second input image based on a difference between a resolution ofthe first input image and a resolution of the second input image.

According to an aspect of an exemplary embodiment, there is provided anapparatus for image sensing including a first image sensor which isconfigured to time-synchronously capture an object, and a second imagesensor configured to time-asynchronously sense a motion event of theobject.

The apparatus may further include a beam splitter which is configured toevenly split a plurality of beams input through a lens such that theevenly split beams are input to the first image sensor and the secondimage sensor, the first image sensor and the second image sensor areseparated by a space.

According to another aspect of an exemplary embodiment, there is alsoprovided an image sensor including a plurality of first image pixelswhich are configured to time-synchronously capture an object, and atleast one second image pixel which is configured to time-asynchronouslysense a motion event of the object. The at least one second image pixelmay be arranged adjacent to at least one corresponding first image pixelamong the plurality of first image pixels.

According to another aspect of an exemplary embodiment, there is alsoprovided an image processing method including acquiring a clock signal,acquiring event data asynchronously occurring in the clock signal,acquiring pixel data synchronously occurring in the clock signal basedon a predetermined number of frames per second; and adjusting the eventdata and the pixel data based on the clock signal.

According to another aspect of an exemplary embodiment, there is alsoprovided an image adjustment method including acquiring a clock signal,acquiring event data which occurs asynchronously in the clock signal,acquiring pixel data which occurs synchronously in the clock signal, andadjusting the event data and the pixel data based on the acquired eventdata and the acquired pixel data.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of an image adjustmentapparatus.

FIG. 2 is a diagram illustrating an example of timing at which eventdata and pixel data is output.

FIGS. 3A through 3C are diagrams illustrating examples of a method oftemporally adjusting event data and pixel data.

FIG. 4 is a diagram illustrating an example of a method of spatiallyadjusting event data and pixel data.

FIG. 5 is a diagram illustrating an example of an image sensingapparatus for providing beams to an event-based image sensor and aframe-based image sensor using a beam splitter.

FIG. 6 is a diagram illustrating an example of an image sensingapparatus disposed in a manner to enable beams penetrating a frame-basedimage sensor to reach an event-based image sensor.

FIGS. 7A and 7B are diagrams illustrating an example of a hybrid imagesensor.

FIG. 8 is a flowchart illustrating an example of an image adjustmentmethod.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Throughout the drawings and the detailed description, unless otherwisedescribed, the same drawing reference numerals will be understood torefer to the same elements, features, and structures. The relative sizeand depiction of these elements may be exaggerated for clarity,illustration, and convenience.

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. Accordingly, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be suggested to those of ordinary skill inthe art. The progression of processing steps and/or operations describedis an example. However, the sequence of and/or operations is not limitedto that set forth herein and may be changed as is known in the art, withthe exception of steps and/or operations necessarily occurring in acertain order. Also, description of well-known functions andconstructions may be omitted for increased clarity and conciseness.

FIG. 1 is a block diagram illustrating an example of an image adjustmentapparatus 100.

Referring to FIG. 1 , the image adjustment apparatus 100 includes areceiver 110 and an adjuster 120.

The receiver 110 may receive a first input image of an object which iscaptured time-synchronously, and a second input image in which a motionevent of the object is sensed time-asynchronously.

For example, the first input image may be output from an image sensorwhich captures the object based on a predetermined number of frames persecond. Further, the second input image may be output from a dynamicvision sensor which captures the object.

The dynamic vision sensor may asynchronously output event data inresponse to sensing a predetermined event. The predetermined event mayinclude a change in brightness of entering beams.

Hereinafter, for descriptive convenience, the image sensor which outputsthe first input image is referred to as a frame-based image sensor, andthe dynamic vision sensor is referred to as an event-based image sensor.

When the event-based sensor senses an event in which brightness of beamsincreases in a predetermined pixel, the event-based image sensor mayoutput an ON event corresponding to the predetermined pixel. Further,when the event-based sensor senses an event in which brightness of beamsdecreases in a predetermined pixel, the event-based sensor may output anOFF event corresponding to the predetermined pixel.

The event-based image sensor may not scan an output of a photodiode ofeach pixel on a frame-by-frame basis. Further, the event-based imagesensor may output pixel data of a portion in which brightness of beamschanges (in contrast to the frame-based image sensor).

A change in brightness of input beams may be based on a motion of anobject. For example, a source of beams may be fixed over time.

In this example, beams reflected by a stationary object may not bechanged. Thus, brightness of beams entering the event-based image sensormay not be changed. In contrast, when an object moves, beams reflectedby the moving object may be changed based on a motion of the object.Thus, brightness of beams entering the event-based image sensor may bechanged.

Since the event-based image sensor outputs event data in response tosensing a predetermined event, the event-based image sensor may outputtime-asynchronous event data.

The frame-based image sensor may synchronously output pixel data basedon a predetermined number of frames per second. For example, theframe-based image sensor may output the pixel data based on a setting,such as 30 frames per second or 60 frames per second.

The frame-based image sensor may output time-synchronous pixel data. Forexample, the frame-based image sensor may be synchronized with a globalclock, and use a frame synchronization signal satisfying thepredetermined number of frames per second. The frame-based image sensormay output pixel data of a corresponding frame in each time interval atwhich the frame synchronization signal indicates “ON”.

The adjuster 120 may adjust the first input image and the second inputimage. Hereinafter, for descriptive convenience, the first input imageis referred to as pixel data and the second input image is referred toas event data.

Adjustment of event data and pixel data may comprehensively include anoperation of matching information included in the event data andinformation included in the pixel data, an operation of generating,maintaining, and managing matching information, an operation ofprocessing at least one of the event data and the pixel data using thematching information, an operation of processing the pixel data based onthe information included in the event data, an operation of processingthe event data based on the information included in the pixel data, etc.

The event-based image sensor may have a high reaction rate or a highresponse rate, and process a relatively small amount of data. Theframe-based image sensor may provide high-resolution image data, andprovide image data of a moving object and a stationary object. Aresponse rate of the event-based image sensor may be measured in unitsless than or equal to microseconds (μs).

The image adjustment apparatus 100 may utilize a characteristic of theevent-based image sensor and a characteristic of the frame-based imagesensor.

For example, the image adjustment apparatus 100 may process the eventdata output from the event-based image sensor having a high responserate, and process the pixel data spatially and temporally adjusted bythe frame-based image sensor. Thus, the image adjustment apparatus 100may make full use of the event data and the pixel data output from theevent-based image sensor and the frame-based image sensor.

The image adjustment apparatus 100 may provide a result of theadjustment to an image processing apparatus. Thus, the image adjustmentapparatus 100 may recognize an object, a character, a face, and a motionof an object. Further, the image processing apparatus may providevarious user interfaces and application services by processing an imageoutput from the image adjustment apparatus 100.

An operation of spatially and temporally adjusting output data of theevent-based image sensor and output data of the frame-based image sensorby the adjuster 120 will be further described with reference to FIGS. 3Athrough 4 .

The event data output by the event-based image sensor may include atleast one of an event timing at which a predetermined event is sensed, apixel index in which the predetermined event is sensed, and an indicatorto indicate a type of the predetermined event. Alternatively, the eventdata may include at least one of information associated with a time atwhich an event is sensed and information associated with an address atwhich a corresponding event is sensed.

For example, the event data may include timing information indicating atime at which an event is sensed based on a global clock. The event datamay also include indexing information indicating a location of a pixelin which the event is sensed. The event data may also includeinformation indicating a type of the sensed event among predeterminedevents.

Although not shown in the drawings, the image adjustment apparatus 100may further include a processing unit. In an exemplary embodiment, theprocessing unit may comprise at least one of a processor, a circuit, ora hardware module.

In response to receipt of the event data, the processing unit mayselectively process the pixel data corresponding to at least one of theevent timing and the pixel index included in the event data.

For example, the event data may include information associated with atiming at which the predetermined event occurs and a location of thepixel in which the corresponding event occurs. The processing unit mayselect the pixel data corresponding to the event timing and the pixelindex included in the event data.

The processing unit may process the pixel data selected from among pixeldata output for each frame period. Thus, the processing unit may reducean amount of data to be processed for performing a predeterminedoperation.

FIG. 2 is a diagram illustrating an example of timing at which eventdata and pixel data is output.

Referring to FIG. 2 , a frame-based image sensor may output pixel data230 using a frame synchronization signal 220 synchronized with a clocksignal 210.

The frame synchronization signal 220 may be synchronized with the clocksignal 210 using a method of matching a rising edge of the framesynchronization signal 220 and a rising edge of the clock signal 210.Also, an ON/OFF period of the frame synchronization signal 220 may bedetermined based on a predetermined number of frames per second of theframe-based image sensor.

The frame-based image sensor may output the pixel data 230 of acorresponding frame in each time interval at which the framesynchronization signal 220 indicates an “ON” state. The frame-basedimage sensor may output the pixel data 230 of a corresponding frame inresponse to the rising edge of the frame synchronization signal 220.

An event-based image sensor may output event data 240 asynchronous tothe clock signal 210. For example, the event-based image sensor maysense a predetermined event and output the event data 240. Thus, a timeinterval 245 between a predetermined rising edge time point of the clocksignal 210 and a time point at which the event data 240 is output maynot be uniform. In other words, the event data 240 is not synchronizedwith the clock signal 210.

Hereinafter, a method of matching the pixel data 230 synchronous to theclock signal 210 and the event data 240 asynchronous to the clock signal210 will be further described with reference to FIGS. 3A through 3C.

FIGS. 3A to 3C are diagrams illustrating examples of a method oftemporally adjusting event data and pixel data.

Referring to FIG. 3A, an event-based image sensor according to anexample embodiment may output asynchronous event data 310, and aframe-based image sensor may output synchronous pixel data 331, 332,333, and 334.

The frame-based image sensor may output the pixel data 331, 332, 333,and 334 in response to frame synchronization timings 321 and 322. Eachof the frame synchronization timings 321 and 322 may include the risingedge of the frame synchronization signal 220 of FIG. 2 .

An image adjustment apparatus according to an example embodiment mayadjust a single piece of the event data 310 to the pixel data 331, 331,333, and 334.

The image processing apparatus according to an example embodiment maymatch an event timing at which the event data 310 is output to one of aplurality of frame periods for the pixel data 331, 332, 333, and 334.

The image processing apparatus may match the event timing to the frameperiod using a variety of methods.

For example, the image processing apparatus may match an event timing atwhich predetermined event data occurs to a subsequent frame period.

In particular, the image processing apparatus may match a first eventtiming of first event data 311 to a first frame period 323 correspondingto a first frame synchronization timing 321. The first framesynchronization timing is an immediately subsequent (e.g., next) framesynchronization timing. The image processing apparatus may adjust thefirst event data 311 to pixel data output in the first frame period 323.

The image processing apparatus may process second event data 312 and aset of third event data 313 using an identical method. The imageprocessing apparatus may adjust the second event data 312 and the set ofthird event data 313 to pixel data of a second frame period 324corresponding to a second frame synchronization timing 322.

The image processing apparatus may process a set of fourth event data314 using an identical method. The image processing apparatus may adjustthe set of fourth event data 314 to pixel data of a frame period (notshown) corresponding to a subsequent frame synchronization timing (notshown).

Referring to FIG. 3B, the image processing apparatus according to anexample embodiment may match an event timing at which a predeterminedevent data occurs to a frame period corresponding to a closest framesynchronization timing.

For example, the image processing apparatus may match the first eventtiming of the first event data 311 to the first frame period 323corresponding to the first frame synchronization timing 321 closest tothe first event timing among the plurality of frame synchronizationtimings. The image processing apparatus may adjust the first event data311 to the pixel data output in the first frame period 323.

The image processing apparatus may process the second event data 312using an identical method. Since the second event timing at which thesecond event data 312 is output is closest to the first synchronizationtiming 321 among the plurality of frame synchronization timings, theimage processing apparatus may adjust the second event data 312 to thepixel data of the first frame period 323 corresponding to the firstsynchronization timing 321.

The image processing apparatus may process the set of third event data313 and the set of fourth event data 314 using an identical method. Aset of third event timings and a set of fourth event timings, at whichthe set of third event data 313 and the set of fourth event data 314 arerespectively output, may be most adjacent to the second framesynchronization timing 322. Accordingly, the image processing apparatusmay adjust the set of third event data 313 and the set of fourth eventdata 314 to the pixel data of the second frame period 324 correspondingto the second frame synchronization timing 322.

Referring to FIG. 3C, the image processing apparatus according to anexample embodiment may match an event timing included in a time intervalcorresponding to a predetermined frame period to the predetermined frameperiod.

For example, the image processing apparatus may match fifth event data341 and a set of sixth event data 342 included in a third frame period361 corresponding to a third frame synchronization timing 351 to thethird frame period 361. The image processing apparatus may adjust thefifth event data 341 and the set of sixth event data 342 to pixel dataoutput in the third frame period 361.

The image processing apparatus may match a set of seventh event data 343included in a fourth frame period 362 corresponding to a fourth framesynchronization timing 352 to the fourth frame period 362. The imageprocessing apparatus may adjust the set of seventh event data 343 topixel data output in the fourth frame period 362.

Although not shown in the drawings, an event-based image sensoraccording to an example embodiment may digitize a timing at which apredetermined event occurs, based on a global clock. For example, whenthe predetermined event occurs, the event-based image sensor may acquirea time at which the corresponding event occurs using the global clock,and output an event signal including time information.

The image processing apparatus may determine a frame synchronizationtiming most adjacent to the corresponding timing or a frame periodincluding the corresponding timing using the digitized timing includedin event data.

FIG. 4 is a diagram illustrating an example of a method of spatiallyadjusting event data and pixel data.

Referring to FIG. 4 , an event-based image sensor and a frame-basedimage sensor according to an example embodiment may support differentresolutions.

For example, to represent an identical area of an object, theevent-based image sensor may use an n_1×n_2-sized pixel matrix 410, andthe frame-based image sensor may use an m_1×m_2-sized pixel matrix 420.

When the event-based image sensor and the frame-based image sensor havedifferent resolutions, pixels of the event-based image sensor may not bematched to pixels of the frame-based image sensor, respectively.

An image adjustment apparatus according to an example embodiment mayadjust event data and pixel data based on a difference between aresolution of the event-based image sensor and a resolution of theframe-based image sensor.

Hereinafter, for descriptive convenience, the resolution of theevent-based image sensor may be lower than the resolution of theframe-based image sensor.

The image processing apparatus according to an example embodiment maymap (m_1/n_1)×(m_2/n_2) pixels of the frame-based image sensor and asingle pixel of the event-based image sensor.

For example, when each of n_1 and n_2 is “4” and each of m_1 and m_2 is“8”, the image processing apparatus may map a single pixel of theevent-based image sensor to four pixels (i.e., 2×2) of the frame-basedimage sensor.

The image processing apparatus may also perform mapping such that arelative location of a pixel 411 included in the pixel matrix 410 maycorrespond to a relative location of a plurality of pixels 421, 422,423, and 424 included in the pixel matrix 420.

For example, the image processing apparatus may map the pixel 411 of theevent-based image sensor to the plurality of pixels 421, 422, 423, and424 of the frame-based image sensor. Therefore, the image processingapparatus may spatially adjust the event data and the pixel data.

FIG. 5 is a diagram illustrating an example of an image sensingapparatus 570 for providing beams to an event-based image sensor and aframe-based image sensor using a beam splitter.

Referring to FIG. 5 , the image sensing apparatus 570 includes anevent-based image sensor 510 and a frame-based image sensor 520.

In the image sensing apparatus 570, the event-based image sensor 510 andthe frame-based image sensor 520 may be spatially separated by a space.The image sensing apparatus 570 further includes a beam splitter 550 tooptically align an input of the event-based image sensor 510 and aninput of the frame-based image sensor 520.

For example, the beam splitter 550 may evenly split beams input througha lens 560 such that the same amount of beams may be input to theevent-based image sensor 510 and the frame-based image sensor 520. Thebeam splitter 550 may provide the evenly split beams to the event-basedimage sensor 510 and the frame-based image sensor 520, simultaneously.In this exemplary embodiment, the lens 560 is a single object lens usedto provide a single image scene, and the beam splitter 550 may be usedto separate the single image scene.

Each of the event-based image sensor 510 and the frame-based imagesensor 520 may be optically aligned by the beam splitter 550 to receivean identical input corresponding to an identical field of view at anidentical point in time.

The image sensing apparatus 570 may include the event-based image sensor510 and the frame-based image sensor 520 packaged independently. Animage adjustment apparatus may input a single image scene captured atthe identical field of view in a space into two image sensor packagesusing the bean splitter 550.

The image sensing apparatus 570 may provide the pixel data and the eventdata to the image adjustment apparatus described with reference to FIGS.1 through 4 .

For example, the image processing apparatus according to an exampleembodiment includes the event-based image sensor 510, the frame-basedimage sensor 520, an adjuster 530, a clock generator 540, the beamsplitter 550, and a lens 560.

Descriptions provided with reference to FIGS. 1 to 4 may be applied tothe event-based image sensor 510, the frame-based image sensor 520, theadjuster 530, and the clock generator 540. Thus, a further detaileddescription will be omitted for conciseness and ease of description.

FIG. 6 is a diagram illustrating an example of an image sensingapparatus 650 disposed for beams penetrating a frame-based image sensor620 to reach an event-based image sensor 610.

Referring to FIG. 6 , the image sensing apparatus 650 includes theevent-based image sensor 610 and the frame-based image sensor 620. InFIG. 6 , the image sensing apparatus 650 does not include the beamsplitter 550 of FIG. 5 .

The image sensing apparatus 650 may use a common structure in which theevent-based image sensor 610 and the frame-based image sensor 620 aresequentially disposed such that beams penetrating the frame-based imagesensor 620 will subsequently reach the event-based image sensor 610 (inlieu of the beam splitter 550 of FIG. 5 ).

For example, the image sensing apparatus 650 may use a structure inwhich the event-based image sensor 610 and the frame-based image sensor620 are laminated.

The image sensing apparatus 650 may be disposed for beams enteringthrough a lens 660 to be input to the frame-based image sensor 620, andfor the beams penetrating the frame-based image sensor 620 to reach theevent-based image sensor 610, due to the laminated structure.

The image sensing apparatus 650 may be provided in a structure in whicha first wafer and a second wafer are laminated. The first wafer mayinclude the frame-based image sensor 620, and the second wafer mayinclude the event-based image sensor 610.

The image sensing apparatus 650 may be manufactured by a process ofbonding the first wafer and the second wafer. The first wafer mayinclude a silicon material and be sufficiently thin to ensurepenetration of beams entering through the lens 660.

While the beams entering through the lens 660 are penetrating the firstwafer, at least a portion of the beams may be absorbed into the firstwafer. For example, a silicon wafer with an energy band gap ofapproximately 1 electron volt (eV) may transmit beams of an infrared rayarea more easily than beams of a visible ray area. The event-based imagesensor 610 included in the second wafer may sense the beams penetratingthe first wafer.

The image sensing apparatus 650 may provide pixel data and event data tothe image adjustment apparatus described with reference to FIGS. 1through 4 .

For example, the image processing apparatus includes the event-basedimage sensor 610, the frame-based image sensor 620, an adjuster 630, aclock generator 640, and the lens 660.

Descriptions provided with reference to FIGS. 1 to 4 may be applied tothe event-based image sensor 610, the frame-based image sensor 620, theadjuster 630, and the clock generator 640 and thus, a further detaileddescription will be omitted.

FIGS. 7A and 7B are diagrams illustrating an example of a hybrid imagesensor 710.

Referring to FIG. 7A, a hybrid image sensor 710 includes a plurality offirst image pixels 712 configured to time-synchronously capture anobject, and at least one second image pixel 711 configured totime-asynchronously sense a motion event of the object.

Hereinafter, for descriptive convenience, a first image pixel isreferred to as a frame-based image pixel, and a second image pixel isreferred to as an event-based image pixel.

In the hybrid image sensor 710, at least one event-based image pixel maybe disposed adjacent to at least one corresponding frame-based imagepixel among a plurality of frame-based image pixels.

Alternatively, each of the at least one event-based image pixel may bedisposed among a plurality of corresponding frame-based image pixels.

For example, a single event-based image pixel may correspond to eightframe-based image pixels. In this case, the event-based image pixel maybe surrounded by the eight frame-based image pixels.

The hybrid image sensor 710 may be manufactured to be a single waferincluding all of the event-based pixels and the frame-based imagepixels.

Since the hybrid image sensor 710 uses the aforementioned hybridstructured-pixels, the hybrid image sensor 710 may not require anoptical alignment or a beam absorption by a wafer.

Referring to FIG. 7B, the hybrid image sensor 710 may provide pixel dataand event data to the image adjustment apparatus described withreference to FIGS. 1 to 4 .

For example, the image processing apparatus according to an exampleembodiment includes the hybrid image sensor 710, an adjuster 730, aclock generator 740, and a lens 720.

Descriptions provided with reference to FIGS. 1 to 4 may be applied tothe event-based image pixels and the frame-based image pixels includedin the hybrid image sensor 710, the adjuster 730, and the clockgenerator 740. Thus, a further detailed description will be omitted forconciseness and ease of description.

FIG. 8 is a flowchart illustrating an example of an image adjustmentmethod.

Referring to FIG. 8 , an image adjustment apparatus according to anexample embodiment acquires a clock signal in operation 810. Inoperation 820, the image adjustment apparatus acquires event dataoccurring asynchronously to the clock signal, in response to sensing apredetermined event.

In operation 830, the image adjustment apparatus acquires pixel dataoccurring synchronously to the clock signal based on a predeterminednumber of frames per second. In operation 840, the image adjustmentapparatus adjusts the event data and the pixel data based on the clocksignal.

Descriptions provided with reference to FIGS. 1 to 7B may be applied tooperations of FIG. 8 . Thus, a further detailed description will beomitted for conciseness and ease of description.

The method according to the above-described embodiments may be recordedin non-transitory computer-readable media including program instructionsto implement various operations embodied by a computer. The media mayalso include, alone or in combination with the program instructions,data files, data structures, etc. Examples of non-transitorycomputer-readable media include magnetic media such as hard disks,floppy disks, and magnetic tape, optical media such as CD ROM discs andDVDs, magneto-optical media such as optical discs, and hardware devicesthat are specially configured to store and perform program instructions,such as read-only memory (ROM), random access memory (RAM), flashmemory, etc. Examples of program instructions include both machine code,such as produced by a compiler, and files containing higher level codethat may be executed by the computer using an interpreter. The describedhardware devices may be configured to act as one or more softwaremodules in order to perform the operations of the above-describedembodiments, or vice versa.

Although embodiments have been shown and described, it would beappreciated by those skilled in the art that changes may be made inthese embodiments without departing from the principles and spirit ofthe exemplary embodiments, the scope of which is defined by the claimsand their equivalents.

What is claimed is:
 1. An image sensor comprising: a single pixel matrixcomprising a plurality of first pixels and a plurality of second pixels,the single pixel matrix being implemented on a single wafer, wherein theimage sensor is configured to generate a first image data based on pixelvalues of the plurality of first pixels and generates a second imagedata based on pixel values of the plurality of second pixels, whereinthe first image data is related to an image of an object, wherein thesecond image data is related a motion event of the object, and wherein aresolution of the first image data is greater than a resolution of thesecond image data.
 2. The image sensor of claim 1, wherein at least oneof second pixels is surrounded by at least some of the plurality offirst pixels.
 3. The image sensor of claim 2, wherein the image sensoris configured to generate the first image data time-synchronously andgenerate the second image data time-asynchronously.
 4. The image sensorof claim 3, wherein N number of second pixels corresponds to M number offirst pixels, and wherein N and M are integer, and wherein N is equal orgreater than 1 and M is greater than N.
 5. The image sensor of claim 4,wherein a single pixel of the plurality of second pixels corresponds toeight first pixels.
 6. The image sensor of claim 4, wherein a singlepixel of the plurality of second pixels corresponds to a matrix of L×Tpixels of first pixels, and wherein the L and T are integer greater than2.
 7. The image sensor of claim 6, wherein the L and T are
 2. 8. Theimage sensor of claim 6, wherein the first image data and the secondimage data are generated at a same frame period.
 9. The image sensor ofclaim 8, wherein the image sensor is configured to generate the firstimage data on a first frame rate or a second frame rate different fromthe first frame rate.
 10. The image sensor of claim 6, wherein thesecond image data includes an event timing at which the motion event ofthe object is sensed.
 11. The image sensor of claim 6, wherein thesecond image data includes an index at which the motion event of theobject is sensed.
 12. The image sensor of claim 9, wherein the imagesensor is configured to transmit the first image data and the secondimage data to a single processing unit.
 13. An image sensor, comprising:a single pixel matrix comprising a plurality of first pixels and aplurality of second pixels, the single pixel matrix being implemented ona single wafer, wherein the image sensor is configured to generate afirst image data based on pixel values of the plurality of first pixelsand generates a second image data based on pixel values of the pluralityof second pixels, wherein the first image data is related to an image ofan object, wherein the second image data is related a motion event ofthe object, and wherein the image sensor is configured to transmit thefirst image data and the second image data to a single processing unit.14. The image sensor of claim 13, wherein a resolution of the firstimage data is greater than a resolution of the second image data. 15.The image sensor of claim 14, wherein the image sensor is configured togenerate the first image data time-synchronously and generate the secondimage data time-asynchronously.
 16. The image sensor of claim 15,wherein a single pixel of plurality of second pixels corresponds to someof the plurality of first pixels.
 17. The image sensor of claim 16,wherein at least one of the plurality of second pixels is surrounded byat least some of the plurality of first pixels.
 18. The image sensor ofclaim 17, wherein the first image data and the second image data aregenerated at a same frame period.
 19. An image sensor, comprising: asingle pixel matrix comprising a plurality of first pixels and aplurality of second pixels, the single pixel matrix being implemented ona single wafer, wherein the image sensor is configured to generate afirst image data based on pixel values of the plurality of first pixelsand generates a second image data based on pixel values of the pluralityof second pixels, wherein the first image data is related to an image ofan object, wherein the second image data is related a motion event ofthe object, wherein the first image data and the second image data aregenerated at a same frame period, and wherein the image sensor transmitsthe first image data and the second image data to a single processingunit.
 20. The image sensor of claim 19, wherein a resolution of thefirst image data is greater than a resolution of the second image data.